1.a program used for searching through websites on the Internet for documents containing a particular word, phrase, or subject
1.One such set of tools are the robots, spiders, and Web crawlers that aggregate information for Internet and intranet search engines.
这种工具有robot、spider和为Internet和Intranet搜索引擎搜集信息的Web爬行榜。
2.Search engines typically disperse Web crawlers to scour the network and compile an index of existing documents.
一般而言，搜索引擎将Web牵引程序分散开以为网络畅通铺平道路，以及对现有文件的索引进行编译。
3.Web crawlers return to each site on a regular basis, such as every month or two, to look for changes.
网络抓取工具定期如每个月或两个查找更改返回到每个站点。
4.Programs called spiders or Web crawlers start by reading pages from a starting list of Web sites.
那些叫做蜘蛛程序或爬行程序的程序开始从网站的起始表读取网页。
5.The advantage of the web crawlers is that they have an extensive database with almost the complete Internet indexed in it.
网络抓取工具的优点是他们几乎完整的网络索引在它有庞大的数据库。